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DoD  Systems  are  Increasingly  Complex... 
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...Systems  of  Systems  (SoS)  even  more  so 


More  and  more,  software  is  the  integrating  element  in  all 

manner  of  systems... 


Architecture-Centric  Army  Acquisition 
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Software  is  Ubiquitous  in  Army  Systems 


Some  systems  are  almost 
exclusively  software 

Systems  like  FBCB2  are  software 
systems,  i.e.,  systems  whose  primary 
functionality  is  derived  totally  (or  nearly 
so)  from  software. 


Software-reliant  systems 
can  be  harder  to  identify 

Systems  like  the  Abrams  tank  are 
software-reliant:  they  depend  on 
software  for  critical  functions  such  as 
navigation,  accurate  fires,  network 
communication,  etc. 
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Coping  with  System/Software  Complexity  is  a 
Must 


2008-2009  Interviews  with  Army  PEOs 

•  Relationship  between  system  engineering  and 
software  engineering  is  driving  system  complexity 

•  Example:  Army  Software  Blocking/Network 
Capability  Sets  -  decade-long  attempt  to  horizontally 
integrate  Battle  Command  software  across  brigade 
elements 

2009  NASA  Study 

•  Software  complexity  leads  to  system  and  operational 
complexity  (and  increases  risk) 

2009  MIT  Study 

•  Software  causes  systems  to  be  become 
“interactively  complex”  (intellectually  unmanageable) 


A  New  Approach  to  Ensuring  Safety  in 
Software  and  Human  Intensive  Systems 


I'lif 


Nancy  G.  Leveson 
MIT 

and 

Safeware  Engineering,  Inc. 
leveson@mit.edu 
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Architecture-Centric  Practices  are  Key... 


Defense  Science  Board  (1994  &  2000) 

•  Software  architecture  techniques  can  reduce  cost  and  cycle  times 

•  Architecture  is  “a  central  theme  for  software  reuse,  product  lines,  and  greater 
exploitation  of  commercial  technology  and  practices” 

Army  Workshop  on  Weapon  Software  Upgrade  Programs  (2001) 

•  Architecture  is  “a  key  technical  focus  for  the  system” 

•  Architecture  is  critical  in  determining  the  future  ability  to  upgrade  the  system 

•  In  2008,  GAO  testimony  noted  similar  findings  for  DoD  business  systems 


NASA  (2009) 

•  “Good  software  architecture  is  the  most  important  defense  against  incidental 
complexity  in  software  designs,  but  good  architecting  skills  are  not  common” 


ACQUIRING  DEFENSE  SOFTWARE 
COMMERCIALLY 


DEFENSE  HI  'SJNEJiS 
TRANSFORMATION 
Sustaining  Progm*. 
Brqulrrs  Continuity  of 
Uwtorship  anti  nn 
tnlrgmlrd  AfUmiarh 
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...But  Acquisition  Practices  Haven’t  Kept  Up  iota 


DoD  Tri-Service  Assessment  Initiative  (2003) 

•  Review  of  21  DoD  program  assessments 

-  poor  software  architecture  practices  are  one  of  the 
systemic  causal  factors  of  software-reliant  systems  issues 

SEI  surveys  and  interviews  of  Army  PMs  and  PEOs  (2004  &  2005) 

•  PMs/PEOs  felt  prime  contractors’  software  architecture 
abilities  were  only  about  average 

-  Yet,  they  also  felt  government  program  office  staffs  were 
not  sufficiently  skilled  to  evaluate  software  architectures 

SEI  analysis  of  results  from  18  architecture  evaluations  (2006) 


Tri-Setvice  Assessment  Initiative 
Phase  2  Systemic  Analysis  Results 


•  >50%  of  the  programs  had  significant  program  risks  driven  by 
lack  of  architecture  training/tools  and  poor  architecture  planning 

•  —2/3  of  risks  discovered  were  risks  of  omission 

-  e.g.,  architectural  decisions  either  not  made  or  not  captured 


Risk  themes  from 
ATAM  data:  preliminary 
results 


Rod  Non! 
Bid  Wood 


Software  Engineering  Institute 
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...But  Acquisition  Practices  Haven’t  Kept  Up  20f3 


On  DoD  projects,  all  too  often  the  SEI  sees 


nt  at  all 


No  architecture  review,  etc. 
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...But  Acquisition  Practices  Haven’t  Kept  Up  sots 


DoD  Chief  Information  Officer  (CIO)  White  Paper  (2008) 

•  3  root  causes  for  architecture  practice  shortcomings  across  the  DoD 

-  Inability  to  leverage  the  benefits  of  an  architecture  due  to  inadequate 
training  on  the  part  of  stakeholders  or  inadequate  communication  on  the 
part  of  architects 

-  Lack  of  incentives  to  encourage  the  professional  growth  of  architects  in  the 
DoD 

-  Lack  of  visibility  into  the  existence  or  value  of  architecture  training 


The  Army  is  aggressively  tackling  these  issues  (and  more)... 
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The  Army  is  Changing  the  Game 


Army  Strategic  Software  Improvement  Program  (ASSIP) 

•  A  partnership  between  the  Office  of  the  Assistant  Secretary  of  the 
Army  for  Acquisition,  Logistics,  and  Technology  (OASA(ALT))  and  the 
Software  Engineering  Institute  (SEI) 

-  Focusing  on  improving  the  Army’s  ability  to  acquire  software-reliant 
systems 

-  Promoting  collaboration  across  the  Army  acquisition  community  and  with 
sister  services 


One  of  ASSIP’s  major  initiatives:  leveraging  software  architecture  in 
acquisition 

•  Education 

•  Application  of  proven  architecture  practices1 

•  Institutionalization:  establishment  of  Chief  Software  Architects 


(CSWAs) 


1  Such  as  found  in  “A  Proactive  Means  for  Incorporating  a  Software  Architecture  Evaluation  in 

a  DoD  System  Acquisition”,  CMU/SEI-2009-TN-004 
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The  Army  is  Building  Expertise 

ASSIP-sponsored  training 

•  Leveraged  SEI  architecture  curriculum 

•  Over  500  Army  personnel  trained 

-  Plus,  over  50  support  contractors  trained 

•  >30%  have  earned  at  least  1  certificate 

-  Software  Architecture  Professional 
-ATAM  Evaluator 
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Participants 


The  Army  is  Sharing  Architecture  Knowledge 


Since  2005,  the  Army  has  held  annual  workshops  on  software 
architecture  and  software  product  lines  to: 

•  explore  relationships  among  enterprise,  SoS,  system,  and  software 
architecture 

•  learn  about  best  practices  and  recent  developments  in  software  architecture 
and  software  product  lines 

•  share  Army  experiences  in  using  software  architecture  and  product  line 
practices  and  how  to  apply  them  effectively  in  an  acquisition  context 

•  understand  issues  regarding  broader  use  of  software  architecture  and 
product  line  practices  in  the  Army 

A  2010  “hands-on”  workshop  is  being  planned  to  broaden  exposure  of 
Army  organizations  to  architecture-centric  practices 

Through  ASSIP,  senior  Army  acquisition  leaders  discuss  software 
issues,  including  architecture,  three  times  per  year 
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The  Army  is  Using  Architecture  Practices 


Since  2002,  14  Army 
projects  have  used 

>the  SEI  Architecture 
Tradeoff  Analysis 
Method®  (ATAM®) 

to  conduct  architecture 
evaluations  and  identify 
architectural  risks  and 
strengths 

and/or 

>the  Quality  Attribute 
Workshop  (QAW) 

to  discover  the  quality 
attributes/non-functional 
requirements  that  drive 
system  design 


Army  Project  (in  alphabetical  order) 

ATAM 

QAW 

Aerial  Common  Sensor 

✓ 

Army  Battle  Command  System 

Command  Post  of  the  Future 

V 

Common  Avionics  Architecture  System 

V 

Distributed  Common  Ground  Station  -  Army 

V 

Force  XXI  Command  Brigade-and-Below 

V 

Future  Combat  Systems 

V 

✓ 

Integrated  Fired  Control 

V 

Joint  Tactical  Common  Operational  Picture  Workstation 

V 

Manned/Unmanned  Common  Architecture  Program 

V 

Network  Operations  Data  Product  Development 
Environment 

✓ 

One  Semi-Automated  Forces 

V 

Sequoyah 

Warfighter  Information  Network  -  Tactical 

V 
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Architecture  Practices  are  Having  an  Impact  i  of  2 

Results  of  2009  survey  of  12  Army  projects  that  employed  ATAM/QAW2 


Artifact  Improvement 

12 

10 


C /) 


Minimal  Moderate  Significant  Very  Substantial 


a  Quality  Attributes  w  Architecture  a  Risks 


•  Most  reported  significant 
improvement  in  their 
architecturally-significant 
artifacts 

•  Architecture  teams  were 
able  to  achieve 
understanding  of 
stakeholder  expectations 
and  the  implications  of 
architectural  decisions  on 
user  needs 


2  Source:  Impact  of  Army  Architecture  Evaluations,  CMU/SEI-2009-SR-007 
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Architecture  Practices  are  Having  an  Impact  2  of 2 


Results  of  2009  survey  of  12  Army  projects  that  employed  ATAM/QAW 

•  Majority  reported  very 
substantial  or  significant 
improvement  in  stakeholder 
communication 


•  Stakeholders,  collectively, 
are  able  to  achieve  a 
common  understanding  of 
the  system  under 
development 

-  Increases  likelihood  that 
product  will  address 
expectations/user  needs 

-  Improves  chances  for 
program  success 


Communication  Improvement 

Very  Substantial 
Significant 
Moderate 

Minimal 

0  2  4  6  8  10  12 

Number  of  Programs 
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The  Army  has  Established  Chief  SW  Architects 


Policy  set  by  the  OASA(ALT)  in  May  2009:  “ All  PEOs  will  appoint  a  Chief 
Software  Architect ’ 

•  CSWAs  will  provide  oversight  and  management  of  software  development 
within  each  PEO’s  portfolio  of  programs 

•  CSWAs  will  provide  guidance  for  software  architecture  design  and  reviews 

-  ensure  consistent  implementation  of  best  practices  and  standards 

-  ensure  systems  engineering  plan  considers  software  engineering  practices 

•  CSWAs  will  complete  the  equivalent  of  the  SEI  software  architecture  course 
series 

ASSIP  is  sponsoring  workshops  to  help  each  CSWA  get  started  and 
develop  a  comprehensive  plan 


The  Army  CSWAs  Have  Set  Their  Own  Goals 


Establish  infrastructures  in  the  PEO  environment  to  support  software 
objectives 

•  Issue  guidance  to  the  PMs  on  software  architecture  requirements 

•  Leverage  the  skills  of  the  systems  and  software  engineers  across  the  organization 

Support  PMs  in  their  software  life-cycle  processes 

•  Monitor  software  architecture  throughout  the  acquisition  life  cycle  to  identify/mitigate 
risks,  link  components  to  business  drivers,  and  focus  on  stakeholder  requirements 

•  Assess  and  evaluate  software  cost  estimates  in  a  system  life  cycle  context 

•  Review  and  endorse  System  Engineering  Plans  (SEPs)  with  the  Chief  System 
Engineer  to  leverage  standards  and  ensure  appropriate  architecture-centric  practices 

Utilize  software  architecture  and  data  interchange  standards  to  minimize 
integration/interoperability  challenges 

•  Ensure  development  of  software  architectures  in  a  system  of  systems  context  to 
address  interoperability  and  manage  software  system  life-cycle 

•  Ensure  program  NR-KPP  are  understood  and  well  defined 

Take  part  in  Communities  of  Interest  (COIs)  across  the  Army  PEO 
portfolio  and  DISA  forums  to  exploit  commonality  and  integration  to  the 
GIG 


Architecture-Centric  Army  Acquisition 
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Institutionalization  is  a  Long  Road 


Pro-active  planning  for  architecture-centric  practices  works  best 

•  the  norm  has  been  reactive,  opportunistic  collaborations  that  lead  to  poor 
cooperation  and  lack  of  follow-through  on  findings 

•  inadequate  planning  leads  to  mis-timed  architecture  evaluations  that  preclude 
achieving  full  benefits 

Establishing  CSWAs  is  a  good  step  toward  institutionalization,  but 
experience  has  shown  that: 

•  developing  an  informed,  comprehensive  approach  to  software  acquisition 
within  a  PEO  organization  will  take  time 

•  striking  a  balance  between  authority  and  influence  is  crucial 

•  having  dual  hats  or  significant  other  responsibilities  will  limit  effectiveness 
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There  Are  Still  More  Opportunities 


Through  ASSIP,  the  CSWAs  may  explore  Army-wide  acquisition 
improvements  such  as: 

•  making  software  architecture  evaluations  standard  practice 

•  requiring  demonstrated  architecture  competency  in  responses  to  system 
acquisition  RfPs 

•  achieving  consensus  on  what  system  and  software  architecture 
documentation  is  most  appropriate  and  cost  effective 

•  increasing  synergy  and  coherence  between  systems  and  software 
engineering  acquisition  practices 


The  CSWAs  &  ASSIP  give  the  Army  a  vehicle  for  instilling 
architecture-centric  practices  in  acquisition 
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Contact  Information 


Stephen  Blanchette,  Jr. 

Deputy  Chief  Engineer,  Army  Programs 
Acquisition  Support  Program 
Telephone:  +1  412-268-6275 
Email:  sblanche@sei.cmu.edu 


U.S.  Mail: 

Software  Engineering  Institute 
Customer  Relations 
4500  Fifth  Avenue 
Pittsburgh,  PA  15213-2612 
USA 


John  Bergey 

Senior  Member  of  the  Technical  Staff 

Research,  Technology,  and  Systems 
Solutions  Program 

Telephone:  +1  215-348-0530 

Email:  ikb@sei.cmu.edu 


Customer  Relations 

Email:  customer- 
relations@sei.cmu.edu 

SEI  Phone:  +1  412-268-5800 

SEI  Fax:  +1  412-268-6257 

World  Wide  Web: 

www.sei.cmu.edu 

www.sei.cmu.edu/contact.html 
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Acronyms 


ATAM 

Architecture  Tradeoff  Analysis 

Method 

NASA 

National  Aeronautics  and  Space 
Administration 

COI 

Community  of  Interest 

PEO 

Program  Executive  Office 

Program  Executive  Officer 

DISA 

Defense  Information  Systems 

Agency 

QAW 

Quality  Attribute  Workshop 

FBCB2 

Force  XXI  Battle  Command  Brigade 
and  Below 

SEI 

Software  Engineering  Institute 

GIG 

Global  Information  Grid 

SoS 

System  of  Systems 
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